Assigning jobs to heterogeneous processing modules

ABSTRACT

A processing system is described which assigns jobs to heterogeneous processing modules. The processing system assigns jobs to the processing modules in a manner that attempts to accommodate the service demands of the jobs, but without advance knowledge of the service demands. In one case, the processing system implements the processing modules as computing units that have different physical characteristics. Alternatively, or in addition, the processing system may implement the processing modules as threads that are executed by computing units. Each thread which runs on a computing unit offers a level of performance that depends on a number of other threads that are simultaneously being executed by the same computing unit.

BACKGROUND

A developer of an online service may design a data center with theobjective of meeting certain performance goals, such as average qualityof response, throughput, and energy efficiency. To this end, thedeveloper will attempt to select an appropriate type of server for usein the data center, and a desired number of these servers. But thisselection task presents complex challenges. Other environments (besidesonline services implemented by data centers) face similar challenges.

SUMMARY

A scheduling module is described herein which assigns jobs toheterogeneous processing modules provided by a processing system. Thescheduling module performs this task based on the service demandsassociated with the jobs, e.g., by attempting to allocateresource-intensive jobs to high-performing processing modules, and lessresource-intensive jobs to lower-performing processing modules. But, inone implementation, the scheduling module performs this task withoutnecessarily having advance knowledge of the resource demands associatedwith the jobs.

According to one illustrative aspect, the scheduling module enables theprocessing system to achieve one or more of: a specified throughputobjective; a specified quality objective; and a specified energyutilization objective. In one case, a quality measure can reflect alevel of completeness of responses provided by the processing modules.

According to another illustrative aspect, the scheduling module includeslogic configured to assign a new job to a processing module having ahighest performance level that is currently available. In anotherimplementation, the scheduling module first assigns the new job to alower-performing processing module; if the job cannot finish by aprescribed time period on the lower-performing processing module, thescheduling module can transfer it to a higher-performing processingmodule.

According to another illustrative aspect, the scheduling module isfurther configured to determine that a previously-assigned job has endedon a newly-vacated processing module having a vacated-module performancelevel. The schedule module then reassigns a most-urgent in-progress jobto the newly-vacated processing module, providing that the in-progressjob is executing on a processing module having a lower performance levelcompared to the vacated-module performance level.

In one implementation, the heterogeneous processing modules correspondto computing units (e.g., processing cores) having different physicalcharacteristics and/or different operating conditions.

In another implementation, the heterogeneous processing modulescorrespond to threads that run on computing units. Each thread whichruns on a computing unit offers a level of performance that depends on anumber of other threads that are simultaneously being executed by thesame computing unit. That is, a thread that runs on a computing unitthat is simultaneously executing M threads has a higher performancelevel than a thread that runs on a computing unit that is simultaneouslyexecuting N threads, providing that M<N.

The above approach can be manifested in various types of systems,components, methods, computer readable storage media, data structures,articles of manufacture, and so on.

This Summary is provided to introduce a selection of concepts in asimplified form; these concepts are further described below in theDetailed Description. This Summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows at least one illustrative processing system havingheterogeneous processing modules. The processing system also includes ascheduling module that is configured to assign jobs to the processingmodules.

FIG. 2 illustrates a scenario in which the scheduling module of FIG. 1assigns jobs to three different processing modules having differentrespective performance levels (e.g., “sizes”).

FIG. 3 is a flowchart which explains one manner of operation of thescheduling module of FIG. 1, corresponding to the example of FIG. 2

FIG. 4 is a graph that conveys one performance-related benefit offeredby the processing system of FIG. 1.

FIG. 5 illustrates a scenario in which the scheduling module uses analternative rule (compared to FIG. 2) for assigning new jobs toheterogeneous processing modules.

FIG. 6 is a flowchart which explains another manner of operation of thescheduling module of FIG. 1, corresponding to the example of FIG. 5.

FIG. 7 is a flowchart which explains one manner of handling interruptsthat affect the jobs.

FIG. 8 illustrates a scenario in which the scheduling module assignsjobs to threads implemented by computing units (e.g., processing cores).Here, the threads constitute instantiations of the heterogeneousprocessing modules of FIG. 1.

FIG. 9 is a flowchart which explains another manner of operation of thescheduling module of FIG. 1, corresponding to the example of FIG. 8.

FIG. 10 shows an illustrative environment in which the type ofprocessing system of FIG. 1 can be employed.

FIG. 11 shows illustrative computing functionality that can be used toimplement any aspect of the features shown in the foregoing drawings.

The same numbers are used throughout the disclosure and figures toreference like components and features. Series 100 numbers refer tofeatures originally found in FIG. 1, series 200 numbers refer tofeatures originally found in FIG. 2, series 300 numbers refer tofeatures originally found in FIG. 3, and so on.

DETAILED DESCRIPTION

This disclosure is organized as follows. Section A provides an overviewof an illustrative processing system that assigns jobs to heterogeneousprocessing modules. Section B describes various implementations of theprocessing system of Section A, such as an implementation in which theprocessing modules correspond to threads that execute on computingunits. Section C describes illustrative computing functionality that canbe used to implement any aspect of the features described in Sections Aand B.

As a preliminary matter, some of the figures describe concepts in thecontext of one or more structural components, variously referred to asfunctionality, modules, features, elements, etc. The various componentsshown in the figures can be implemented in any manner by any physicaland tangible mechanisms, for instance, by software, hardware (e.g.,chip-implemented logic functionality), firmware, etc., and/or anycombination thereof. In one case, the illustrated separation of variouscomponents in the figures into distinct units may reflect the use ofcorresponding distinct physical and tangible components in an actualimplementation. Alternatively, or in addition, any single componentillustrated in the figures may be implemented by plural actual physicalcomponents. Alternatively, or in addition, the depiction of any two ormore separate components in the figures may reflect different functionsperformed by a single actual physical component. FIGS. 10 and 11, to bedescribed in turn, provide additional details regarding one illustrativephysical implementation of the functions shown in the figures.

Other figures describe the concepts in flowchart form. In this form,certain operations are described as constituting distinct blocksperformed in a certain order. Such implementations are illustrative andnon-limiting. Certain blocks described herein can be grouped togetherand performed in a single operation, certain blocks can be broken apartinto plural component blocks, and certain blocks can be performed in anorder that differs from that which is illustrated herein (including aparallel manner of performing the blocks). The blocks shown in theflowcharts can be implemented in any manner by any physical and tangiblemechanisms, for instance, by software, hardware (e.g., chip-implementedlogic functionality), firmware, etc., and/or any combination thereof.

As to terminology, the phrase “configured to” encompasses any way thatany kind of physical and tangible functionality can be constructed toperform an identified operation. The functionality can be configured toperform an operation using, for instance, software, hardware (e.g.,chip-implemented logic functionality), firmware, etc., and/or anycombination thereof.

The term “logic” encompasses any physical and tangible functionality forperforming a task. For instance, each operation illustrated in theflowcharts corresponds to a logic component for performing thatoperation. An operation can be performed using, for instance, software,hardware (e.g., chip-implemented logic functionality), firmware, etc.,and/or any combination thereof. When implemented by a computing system,a logic component represents an electrical component that is a physicalpart of the computing system, however implemented.

The phrase “means for” in the claims, if used, is intended to invoke theprovisions of 35 U.S.C. §112, sixth paragraph. No other language, otherthan this specific phrase, is intended to invoke the provisions of thatportion of the statute.

The following explanation may identify one or more features as“optional.” This type of statement is not to be interpreted as anexhaustive indication of features that may be considered optional; thatis, other features can be considered as optional, although not expresslyidentified in the text. Finally, the terms “exemplary” or “illustrative”refer to one implementation among potentially many implementations

A. Overview of an Illustrative Processing System

FIG. 1 shows an illustrative processing system 102 having heterogeneousprocessing modules 104. The processing system 102 can be used in anyenvironment. For example, a data center may use a group of processingsystems 106 to provide any type of service or combination of services toend users. In that scenario, the processing system 102, which is one ofthe group of processing systems 106, may correspond to a server. Inanother case, the processing system 102 may represent functionalityprovided by any type of local computer device.

Each processing module in the collection of processing modules 104includes logic for performing processing operations. Section B describesdifferent physical implementations of the processing modules 104. Asexplained there, the processing system 102 can implement the processingmodules 104 as processing cores of one or more CPUs. Alternatively, theprocessing system 102 can implement the processing modules 104 asthreads which run on the processing cores. Still other implementationsare possible.

The processing modules 104 are heterogeneous because they offerdifferent levels of performance. For example, at least two of theprocessing modules 104 may operate at different speeds. A processingmodule that runs at a relatively fast rate can execute more operationscompared to a processing module that runs at a slower rate. However, aprocessing module that runs at the faster rate may also consume moreenergy than its lower-performing counterpart.

FIG. 1 shows a high-level example in which the processing modules 104employ three different types of processing modules having threerespective performance levels. The collection of processing modules 104can include zero, one, or more members from each of these groups ofprocessing modules. Other implementations can offer just two types ofprocessing modules, or more than three types of processing modules.

In the example of FIG. 1, the processing modules 104 include a firstgroup 108 of processing modules, each of which offers the lowest levelof performance. In other words, each of the processing modules in thisgroup 108 may run the slowest, but may consume the least amount ofenergy during its operation. In a third group 110, each processingmodule offers the highest level of performance. In other words, each ofthe processing modules in this group 110 may run the fastest, but mayconsume the most amount of energy. A processing module in a second group112 exhibits a level of performance and energy expenditure midwaybetween a processing module of the first group 108 and a processingmodule of the third group 110. From a metaphorical standpoint, the firstgroup 108, second group 112, and third group 110 may provide,respectively, “small,” “medium,” and “large,” processing modules.

The processing system 102 may also include job scheduling functionality114 that assign jobs to individual respective processing modules in thecollection of processing modules 104. The job scheduling functionality114 can be implemented as a hardware logic component, a softwarecomponent, or some combination thereof. For example, the processingsystem 102 can implement the scheduling functionality 114 as anapplication-level or operating system level program module, or someother type of program module. In another implementation, a componentwhich is external to the processing system 102 can implement the jobscheduling functionality 114, in whole or in part. In oneimplementation, the job scheduling functionality 114 can be subject tointerrupts generated by any logic component associated with theprocessing system 102, such as an operating system.

In one implementation, the job scheduling functionality 114 can includea job receiving module 116, a scheduling module 118, a job queue 120,and a result output module 122. In operation, the job receiving module116 receives jobs from any job source(s) 124. For example, the jobreceiving module 116 may represent interface functionality provided by adata center which implements one or more online services. In thisenvironment, the job source(s) may correspond to end users. The usersoperate user devices to submit job requests to the job receiving module116. Next, the scheduling module 118 assigns the jobs to the processingmodules 104 based on the rules described below. The processing modules104 then perform their work on the jobs to provide output results. Theresult output module 122 forwards the output results to one or morerecipients 126. For example, the recipients 126 may correspond to theusers who submitted the requests.

To be more concrete, consider a first non-limiting scenario in which theprocessing system 102 is part of the functionality used by an onlinesearch engine. In that environment, the job receiving module 116 canreceive a query from a user. That query represents a job requestassociated with a new job. The search engine can then use a plurality ofsearch modules to process the query. More specifically, each searchmodule operates by comparing the query against a portion of an invertedindex to identify documents (if any) which match the query. Anaggregator module compiles partial results received from the searchmodules to form a final result. The result output module 122 then sendsfinal result to the user, e.g., as a list of documents that match thequery.

In this context, the processing system 102 may represent one serveramong many servers employed by the search engine. The processing modules104 on the processing system 102 correspond to a subset of the searchmodules employed by the search engine. In this case, the schedulingmodule 118 assigns queries to the search modules.

Consider a second non-limiting scenario in which the processing system102 is part of the functionality used by an online financial analysisservice. In that environment, the job receiving module 116 can receive arequest from a user to perform a financial calculation. The financialmodeling engine can use a plurality of analysis modules to perform thecalculation. More specifically, in one merely illustrative case, eachanalysis module performs a sampling operation, e.g., using Monte Carlosimulation or some other statistical estimation technique, to generatean output result. The result output module 122 returns the output resultto the user.

In this context, the processing system 102 may again represent oneserver among many servers employed by the financial analysis service.The processing modules 104 on the processing system 102 correspond to asubset of the analysis modules used by the financial analysis service.In this case, the scheduling module 118 assigns requests to the analysismodules.

The scenarios described above are cited by way of illustration, notlimitation. Still other applications are possible.

In many cases, a job may be said to have a starting time, a deadline,and a service demand. The starting time refers to the time at which thescheduling module 118 receives the job. The deadline may correspond tothe time at which the user demands a response to the job. In some cases,the deadline is explicit, and is communicated in the job request. Inother cases, the deadline is implicit. For example, a search engine mayendeavor to deliver a response within a specified time, e.g., a fractionof a second in one case, or one or more seconds in another case. Butthat deadline is not explicitly expressed in each query. The servicedemand refers to the amount of processing resources that will berequired to perform the job in the specified period of time.

In some cases, a job request will explicitly convey the service demandassociated with a job, or the processing system 102 can otherwisedetermine the service demand associated with the job. For example, inthe search context, it may be possible, in some circumstances, toestimate the processing demand of a particular query if that same queryhas been processed on one or more prior occasions, and/or based on otherfactors. Further, the processing system 102 can generate statistics,based on historical information, pertaining to the expected distributionof processing loads associated with queries. In the search context, forinstance, the processing system 102 can determine that most jobs aretypically “short” in duration.

In other cases, however, the amount of resources that will be requiredto perform the job will be unknown or at least uncertain. In thisscenario, the processing system 102 only discovers the requirements ofthe job in the course of processing the job. In general, this means thatthe scheduling module 118 is tasked with the responsibility ofallocating jobs to processing modules 104 without sometimes knowing thedemands which each job will impose during its execution. If theprocessing demand is known or capable of being estimated, thisinformation can contribute to the level of urgency that is assigned tothe job; but, as a default principle, the demand is considered to beunknown.

In some cases, the processing modules 104 can deliver a response thathas high quality. Consider the search context. Each processing modulemay have sufficient time to match a query against its entire invertedindex portion. As a whole, the processing modules 104 can thereforereturn a result which identifies all the documents which can possiblymatch the query. In other cases, the processing modules 104 may delivera response that has less-than-optimal quality. For example, a processingmodule may have time to match the query against only a part of itsinverted index, not the entire index. As a result, the processingmodules 104, as a whole, may return a result which is not a completereflection of the all the documents which can possibly match the query.The result in this case may be satisfactory because the processingmodules can be configured to identify documents in the order of mostrelevant to least relevant. Thus, even if the search is truncated beforefully completing, the less-than-optimal results may identify the mostrelevant documents. But the results are nonetheless not optimal. Inanother example, a financial analysis service may use processing modulesthat perform any type of iterative calculations to arrive at a result.The accuracy of that result increases with the amount of time that theprocessing modules are given to perform their calculations.

FIGS. 2-7 provide details regarding the logic that the scheduling module118 can use to schedule jobs. In general, the scheduling module 118 canbe said to assign jobs to the processing modules 104 based on theservice demands associated with those jobs. More specifically, thescheduling module 118 may attempt to assign resource-intensive jobs tohigh-performing processing modules, while less resource-intensive jobsto lower-performing jobs. But as noted above, the scheduling module 118may be forced to perform this allocation without advance knowledge ofthe service demands of the jobs.

Overall, the processing system 102 may attempt to achieve one or moreperformance objectives, given one or more specified constraints. In onecase, the scheduling module 118 attempts to provide the highest qualityresults possible (e.g., on average), subject to one or more constraints(e.g., an energy utilization budget, etc.). Different environments canassess quality in different ways. In the search context described above,the quality of a response reflects the time-dependent completeness ofthe response. In the financial analysis context, the quality of aresponse reflects the time-dependent accuracy of the response. In othercases, the scheduling module 118 can primarily attempt to increasethroughput, subject to one or more constraints. Throughput refers to anamount of jobs that can be processed by the processing system 102 in aprescribed amount of time. In other cases, the job schedulingfunctionality 114 can primarily attempt to lower energy utilization,subject to one or more other constraints. Energy efficiency refers tothe amount of energy consumed by the processing system 102 in performingits operations. In other cases, the scheduling functionality 114 canprimarily attempt to reduce the number of processing modules that areused by a data center, subject to various constraints, and so on. Theseperformance goals are not mutually exclusive. That is, in many cases, adesigner will have achieved two or more of the performance goals in aprincipal attempt to achieve one of the goals.

Starting with FIG. 2, this figure illustrates a scenario in which thescheduling module 118 assigns jobs to three different processing moduleshaving respective performance levels. The processing module A has thelowest performance level, the processing module C has the highestperformance level, and the processing module B has a performance levelthat is between the levels of modules A and B. The performance levelsmay correspond to the speeds at which the processing modules can performtheir operations. That is, the processing module A is the slowest, butconsumes the least energy, while the processing module C is the fastest,but consumes the most energy. The processing modules A, B, and C can beinformally referred to as small, medium, and large modules,respectively.

FIG. 2 annotates operations with time information (t₁, t₂, etc.) thatconveys an illustrative ordering of the operations. At time t₁, thescheduling module 118 assigns a new job J₁ to the processing module withthe highest-performing module that is currently available. Thehighest-performing module at this time is processing module C. At timet₂, the scheduling module 118 assigns a new job J₂ to thehighest-performing module that is currently available. Since processingmodule C is now unavailable, the highest-performing module at this timeis processing module B. At time t₃, the scheduling module 118 assigns anew job J₃ to the highest-performing module that is currently available.The only processing module that is available at this time is processingmodule A, so the scheduling module assigns job J₃ to processing moduleA.

At time t₄, assume that the scheduling module 118 receives a fourth jobJ₄. All of the processing modules (A, B, and C) are currently processingjobs, so none of them are available to accept a new job. In view ofthis, the scheduling module 118 adds the new job J₄ to the job queue120.

At time t₅, assume that the job J₁ that is running on the processingmodule C ends, e.g., either because its work is done, or because itsallotted processing time has expired, or because it has been suspended,or because it terminates for any other reason. At time t₆, thescheduling module 118 responds to this event by transferring themost-urgent in-progress job from a “smaller” processing module to theprocessing module C. In one case, the urgency of a job corresponds, inpart, to its age, measured from the time it was received by thescheduling module 118. Thus, the scheduling module 118 will transfer thejob J₂ to the processing module C, since the job J₂ is the oldestin-progress job.

The migration of job J₂ to the processing module C leaves the processingmodule B idle. In view thereof, at time t₇, the scheduling module 118applies the same rule described above, e.g., by transferring themost-urgent in-progress job from a smaller processing module to theprocessing module B. This rule prompts the scheduling module 118 to movethe job J₃ to the processing module B.

The migration of job J₃ to the processing module B leaves the processingmodule A idle. There are no more in-progress jobs on smaller processingmodules to move to larger processing modules because processing module Ais the smallest processing module. But there is a job J₄ in the jobqueue 120. Therefore, at time t₈, the scheduling module 118 moves thejob J₄ to the processing module A. FIG. 2 represents the final outcomeof the operations described above at a time t₉.

In general, note that the scheduling module 118 acts to move jobs fromsmaller processing modules to larger processing modules. In oneimplementation, the scheduling module 118 will not move jobs betweenprocessing modules of the same size. Nor will it move jobs fromhigher-performing processing modules to lower-performing processingmodules. The movement of jobs from lower to higher-performing processingmodules increases the likelihood that a resource-intensive job willcomplete by its deadline, or that this job can at least generate ahigher quality result (if it cannot complete by its deadline). This isbecause the higher-performing processing module can devote moreresources to the migrated job, compared to the lower-performingprocessing module. An “old” job is treated as a likelyresource-intensive job because it has already consumed a significantamount of processing resources, and it is expected to consume even moreresources. In other words, the scheduling module 118 estimates theservice demand of a job by observing how much resources it consumesduring its execution; if the job fails to complete quickly, the job isclassified as a resource-intensive job.

FIG. 3 is a procedure 300 which generalizes the operations of FIG. 2 inflowchart form. In block 302, the scheduling module 118 determineswhether a new job has been received. In block 304, the scheduling module118 determines whether a processing module is currently available. Ifso, in block 306, the scheduling module 118 assigns the new job to thehighest-performing processing module that is currently available. Ifnot, in block 308, the scheduling module 118 stores the new job in thejob queue 120.

In block 310, the scheduling module 118 determines whether a job that isrunning on one of the processing modules has terminated for any reasondescribed above. This termination leaves a newly-vacated processingmodule. In block 312, the scheduling module 118 determines whether thereis an in-progress job on a lower-performing processing module (e.g., asmaller processing module). If so, in block 314, the scheduling module118 moves the in-progress job to the newly-vacated module. FIG. 3indicates that blocks 312 and 314 can be repeated one or more timesuntil no in-progress job can be migrated to a higher-performingprocessing module. This operation eventually leaves a feed-up processingmodule.

In block 316, after the above-described migration of in-progress jobs,the scheduling module 118 determines whether there are any jobs in thejob queue 120. If so, in block 318, the scheduling module 118 moves themost-urgent (e.g., the oldest) job from the job queue 120 into thefreed-up processing module.

FIG. 4 is a graph that conveys one performance-related benefit offeredby the processing system 102 of FIG. 1, which implements the schedulinglogic shown in FIGS. 2 and 3. In particular, the curve 402 shows thequality of results provided by the processing system 102 versus thequantity of traffic processed by the processing system 102. Morespecifically, the curve 402 indicates that the quality of the processingsystem 102 slightly degrades with an increase in traffic. Quality can beassessed in any manner described above. For example, quality may reflectthe level of completeness of the responses.

FIG. 4 also provides two other curves (404, 406) which describe theperformance of two baseline cases. In a first case, a high-performancehomogeneous processing system is constructed using a homogeneous set ofhigh-performance processing modules, e.g., only “big” processingmodules. The curve 404 represents the performance of this type ofsystem. The curve 404 indicates that the high-performance homogeneousprocessing system offers good quality for low traffic volumes, but thequality quickly degrades as load increases. In a second case, alow-performance homogeneous processing system is constructed using ahomogeneous set of lower-performance processing modules, e.g., only“small” processing modules. The curve 406 represents the performance ofthis type of system. The curve 406 indicates that the lower-performancehomogenous processing system offers performance that degrades onlyslightly with load, but that the quality is overall lower compared tothe high-performance processing system at small loads.

In contrast, the curve 402 indicates that the heterogeneous processingsystem 102 avoids the steep drop-off in quality levels exhibited bycurve 404. At the same time, the heterogeneous processing system 102maintains considerably higher quality levels compared to the curve 406.In other words, the heterogeneous processing system 102 combines themerits of the high-performance and lower-performance homogeneousprocessing systems. The improvement in quality may also be accompaniedby improvement in throughput and energy efficiency, relative to thehigh-performance homogeneous processing system and the lower-performancehomogeneous processing system.

FIG. 5 illustrates a scenario in which the scheduling module 118 uses analternative rule (compared to FIGS. 2 and 3) for assigning new jobs tothe processing modules 104. That is, in the case of FIG. 2, thescheduling module 118 assigns each new job to the processing module withthe highest performance level that is currently available. In the caseof FIG. 5, the scheduling module 118 assigns a new job to the processingmodule with the lowest performance level. But if that new job fails tofinish on the low-performance processing module within a prescribedamount of time (and/or as per any other execution constraint), thescheduling module 118 will migrate it to a higher-performing processingmodule, such as the processing module with the highest performance levelthat is currently available.

More specifically, at time t₁, both the processing modules A and B areidle. The scheduling module 118 moves a new job J₁ to the processingmodule A because that module is smaller (lower-performing) compared tothe processing module B. Time t₂ represents a predetermined intervalfollowing time t₁. At time t₂, the scheduling module 118 moves the jobJ₁ to the processing module B because it has failed to complete while onprocessing module B. Time t₃ reflects the outcome of the movement attime t₂.

The policy illustrated in FIG. 5 helps reduce the power consumption ofthe processing system 102. That is, the policy allows a job to completeon a low-performance (and low-powered) processing module if, in fact,the job is not a particularly resource-intensive job. This saves thepower that would be otherwise “wasted” if a short job were immediatelydeployed to a higher-performing processing module. If the job proves tobe resource-intensive, the scheduling module 118 can move it to ahigher-performing processing module in the manner described above,incurring only a slight processing delay due to the transfer.

FIG. 6 is a procedure 600 which explains the operations shown in FIG. 5in flowchart form. In block 602, the scheduling module 118 determineswhether a new job has been received. If so, in block 604, the schedulingmodule 118 determines whether there are any processing modules that arecurrently available on which to place the new job. If so, in block 606,the scheduling module 118 places the new job on the lowest-performingprocessing module that is currently available. If there are noprocessing modules available, then, in block 608, the scheduling module118 places the new job in the job queue 120. In block 610, thescheduling module 118 determines whether there is an in-progress job ona lower-performing processing module that has a level of urgency whichexceeds a prescribed urgency threshold (U_(thresh)). For example, thescheduling module 118 can perform this test by determining whether anyin-progress job has been running on a lower-performing processing modulefor more than a prescribed amount of time without completing. If so, inblock 612, the scheduling module 118 determines whether there is ahigher-performing processing module that is available. And if so, inblock 614, the scheduling module 118 moves the in-progress job to thehighest-performing processing module that is currently available.

The logic shown in FIG. 6 can be used as a stand-alone processingprotocol, or in conjunction with the logic shown in FIG. 3. For example,in the latter case, the scheduling module 118 can assign a new job tothe smallest processing module that is currently available. Ifappropriate, the scheduling module 118 can then move the newly-placedjob to a larger processing module if it fails to complete by aprescribed time. Simultaneously with the above-described process, thescheduling module 118 can handle the termination of in-progress jobsaccording to the logic described in blocks 310-318 of FIG. 3. This meansthat the scheduling module 118 can move in-progress jobs in twocircumstances, e.g., based on a first rule described in blocks 610-614of FIG. 6 and a second rule described in blocks 310-318 of FIG. 3.

In view of the alternative protocol of FIG. 6, a statement herein thatindicates that a new job is moved to a highest-performing processingmodule is broad enough to encompass at least two scenarios. In a firstcase, the scheduling module 118 may immediately transfer the new job tothe highest-performing processing module. But in a second case, thescheduling module 118 may transfer the new job to the highest-performingprocessing module after first deploying the job on a lowest-performingprocessing module.

FIG. 7 is a procedure 700 which explains one manner in which thescheduling module 118 may handle interrupts. The interrupts mayoriginate from any source, such as an operating system, an application,etc., and they may be issued for any environment-specific reason. In onecase, the interrupt is directed to a single identified job, and has theeffect of suspending the execution of that job. In another case, theinterrupt is directed to plural jobs, such as an identified class ofjobs, etc. Here, the interrupt has the effect of suspending theexecution of the plural affected jobs. Nevertheless, to simplify theexplanation, it will henceforth be assumed that each interrupt isdirected to a single job.

In block 702, the scheduling module 118 determines whether an interrupthas been received. If so, the scheduling module 118 determines whetherthe affected job is currently being executed by a processing module. Ifthis is true, then, in block 704, the scheduling module 118 can stop thejob, mark it to indicate that it is “not ready” for processing, andplace the job in the job queue 120. Alternatively, the scheduling module118 can place the marked job in another queue (not shown) that isdedicated to suspended jobs. Suppose, alternatively, that the interruptpertains to a job that is already in the job queue 120. The schedulingmodule 118 can therefore mark this job to indicate that it is not readyfor processing. Accordingly, at any given time, the job queue 120 cancontain a mix of jobs, some of which are “ready” for scheduling, andothers which are “not ready” for scheduling (that is, assuming that asingle queue is used to handle all job requests). The scheduling module118 will not schedule any job that is marked as “not ready.”

In block 706, the scheduling module 118 determines whether anotification has been received that indicates that the interrupt hasbeen serviced. This means that the processing of the affected job canresume. In block 708, the scheduling module 118 determines whether thedeadline of the affected job has expired. If this is true, then, inblock 710, the scheduling module 710 terminates the job. In anotherimplementation, the scheduling module 118 can also terminate the job ifthe amount of time that is left to perform the job is below a prescribedthreshold. The processing system 102 can optionally send the results ofwhatever processing it has performed on the terminated job, prior to theinterrupt, to the recipient 126.

If the job has not expired, then, in block 712, the scheduling module118 schedules the jobs on a processing module in accordance with itslevel of urgency. The level of urgency may reflect the time at which thejob was originally received, and/or any other appropriateurgency-related factor(s).

The scheduling module 118 can use different approaches to carry outblock 712. In a first approach, the scheduling module 118 can assign theresumed job to the highest-performing processing module that iscurrently available, with or without the preliminary processingdescribed in FIG. 6. The scheduling module 118 can then rely on themigration logic described in FIG. 3 to potentially move the resumed jobto a higher-performing processing module upon the termination of anotherjob. Thus, if the resumed job is truly the most urgent job, it maymigrate to a higher-performing processing module in due course as otherjobs end.

In a second approach, the scheduling module 118 can rank the urgencylevels of the jobs that are currently running on the processing modules,together with the urgency level of the resumed job. The schedulingmodule 118 can then migrate jobs to processing modules that are suitedto the respective urgency levels of the jobs. This operation willeffectively create a vacancy having an appropriate performance capacityto receive the resumed job. For example, assume that the resumed job isthe most urgent of all jobs. The scheduling module 118 can migrate oneor more existing jobs to leave a vacancy on the highest-performingprocessing module that is currently available. The scheduling module 118can then assign the resumed job to that processing module. Incidentally,the scheduling module 118 can also perform this kind of proactiveprocessing in any circumstance in which it receives a new job that ismarked as having a higher level of urgency than one or more existingjobs that are being run on the processing modules (where urgency can beassessed based on any factor or combination of factors, not just basedon the time that the job was received).

The above two approaches to scheduling assume that there is at least oneavailable processing module to receive the resumed job. If there are novacancies, the scheduling module 118 can assign the resumed job to thejob queue 120. Alternatively, the scheduling module 118 can optionallysuspend a lowest-performing job, and then place that suspended job backon the job queue 120. The scheduling module 118 can then place theresumed job on a vacant processing module, with or without firstmigrating jobs on the processing modules to free up a processing modulehaving an appropriate performance capacity.

Still other implementations are possible to handle interrupts. Further,the scheduling module 118 can also handle some interrupts by justsuspending the affected jobs, and without removing these jobs from theirrespective processing modules. When the interrupts are serviced, thejobs resume on their respective processing modules.

As a final topic in this section, note that FIG. 1 indicates that thescheduling module 118 can also optionally interact with other processingsystems, as per interaction 128. For example, the scheduling module 118can also assign jobs to the processing modules (not shown) of one ormore other processing systems. Further, the scheduling module 118 canaccept jobs from one or more other processing systems.

Different rules can be used to govern an inter-system job assignment. Inone case, the scheduling module 118 can determine whether its localprocessing system has insufficient capacity to satisfactorily processthe jobs it has received (and/or the job it expects to receive) withindesired performance goals. For example, the scheduling module 118 canreach this conclusion when its job queue 120 accumulates a prescribednumber of jobs. In this circumstance, the scheduling module 118 candetermine whether one or more other processing modules have sparecapacity to accept new jobs. If spare capacity exists, the schedulingmodule 118 can then transfer a portion of yet-to-be assigned jobs and/orin-progress jobs to the processing modules of one or more otherprocessing systems. The scheduling module 118 may likewise accept jobsforwarded by other processing systems (where each other processingsystem may perform the same decision process described above).

For the above reason, the collection of heterogeneous processing modules104 shown in FIG. 1 can be considered as encompassing a first group ofprocessing modules that are physically implemented by the processingsystem 102, together with a second group of processing modules that areimplemented by one or more other processing systems, and thereforerepresent virtual members of the collection of processing modules 104.

In yet another implementation, a data center can employ a centralscheduling module (not shown) which assigns jobs to the processingmodules provided by all of the processing systems in the data center. Inthis context, the data center itself may be considered as one largeprocessing system having an aggregate collection of heterogeneousprocessing modules; each individual processing system contributesprocessing modules to the master collection of processing modules. Thecentral scheduling module can be used instead of the individualscheduling modules provided by individual processing systems, or it cansupplement the functions of the individual scheduling modules. Forexample, the central scheduling module can transfer jobs from oneprocessing system to another, but each individual scheduling module canotherwise perform the scheduling logic described in FIGS. 2-7.

B. Illustrative Implementations of the Processing Modules

Section A described the processing modules principally in the context oftheir performance. That is, that section described a large processingmodule as logic that performs computational tasks at a relatively highperformance level, such as a high processing speed. Likewise, Section Adescribed a small processing module as logic that performs computationaltasks at a lower performance level. This section describes differentways that the processing system 102 may physically implement thecollection of heterogeneous processing modules 104.

In one case, the processing modules 104 correspond to computing units ofany type that have different physical characteristics. For example, theprocessing modules 104 may correspond to processing cores of a serverthat have different physical characteristics or are run under differentphysical conditions. For example, two computing units may physicallydiffer because they incorporate different hardware components.Alternatively, or in addition, two computing units may physically differbecause they incorporate different processing architectures.Alternatively, or in addition, two computing units may differ becausethey incorporate different software components. Alternatively, or inaddition, two computing units may physically differ because they are runat different speeds, or power budgets, etc. Alternatively, or inaddition, two computing units may differ because they have access todifferent sets of supporting resources, and so on. No limitation isplaced on the manner in which two computing units can be designed and/oroperated to offer different levels of performance.

In another case, a processing module may correspond to a thread that isimplemented by a computing unit, such as a hardware thread which isimplemented by a processing core that is capable of performingsimultaneous multi-threading (SMT), also known as hyperthreading. In onecase, for example, each computing unit can perform two-way SMT byincorporating two hardware threads. In another case, each computing unitcan perform four-way SMT by incorporating four hardware threads, and soon. Or the collection of processing modules 104 can include computingunits that support different numbers of hardware threads.

In this implementation, each thread which runs on a computing unitoffers a level of performance that depends on a number of other threadsthat are simultaneously being executed by the same computing unit. Thatis, a thread that runs on a computing unit that is simultaneouslyexecuting M threads has a higher performance level than a thread thatruns on a computing unit that is simultaneously executing N threads,providing that M<N. This relationship is due to the fact that, in someplatforms, the complexity of processing is reduced with a decrease inthe number of executing threads. Further, the competition for a sharedset of resources goes down with a decrease in the number of executingthreads.

FIG. 8 illustrates a scenario in which the scheduling module 118 assignsjobs to hardware threads implemented by two computing units (CU₁ andCU₂), each of which implements two-way SMT. That is, the first computingunit CU₁ has two threads (A₁ and A₂), while the second computing unitCU₂ likewise has two threads (B₁ and B₂).

At time t₁, the scheduling module 118 attempts to assign a new job J₁ tothe highest-performing processing module (e.g., the highest-performingthread in this case). The scheduling module 118 performs this task byassigning the job J₁ to an idle computing unit, if one is available. Anidle computing unit is a computing unit that is not running any jobs onits threads. Here, at time t₁, the scheduling module 118 assigns the jobJ₁ to thread A₁ of computing unit CU₁, because that computing unit isidle. If both computing units are idle, the scheduling module 118 canselect one of them based on any selection criterion, such as by randomlyselecting a computing unit.

At time t₂, assume that the scheduling module 118 receives a second jobJ₂. The scheduling module 118 will apply the same rule described above,which results in assigning job J₂ to thread B₁ of the computing unitCU₂. This is because the computing unit CU₂ is idle, and thus each ofits threads represents a highest-performing processing module.

At time t₃, next assume that the scheduling module 118 receives a thirdjob J₃. The scheduling module 118 will attempt to apply the same ruledescribed above. But there are no more empty computing units at thistime. Therefore, the scheduling module 118 will identify the highestperforming thread that is running on a computing unit, where thatcomputing unit is simultaneously executing at least one other thread. Ifthere are two or more computing units that offer a thread that meetsthis criterion, then the scheduling module 118 will choose the computingunit that is handling the least-urgent in-progress job (e.g., theyoungest job). In the particular context of FIG. 8, the schedulingmodule 118 assigns the job J₃ to the thread B₂ of the computing unitCU₂. Note that both CU₁ and CU₂ offer threads (A₂ and B₂, respectively)having the same performance level. But the job J₁ that is being run onthread A₁ is older than the job J₂ that is being run on thread B₁.Therefore, job J₂ is less urgent that job J₁. For this reason, thescheduling module 118 adds the job J₃ to the thread B₂, not the threadA₂.

Next assume that, at time t₄, the job J₁ terminates on thread A₁ for anyreason stated above. The scheduling module 118 handles this event byidentifying the most urgent job being handled by a lower-performingthread (compared to the thread A₁ that is newly-vacated). In the case ofFIG. 8, the computing unit CU₁ now is running no jobs on its threads,while the computing unit CU₂ is running two jobs on its two respectivethreads. Therefore, both the threads B₁ and B₂ on CU₂ representlower-performing threads compared to the idle thread A₁ of CU₁. Withrespect to CU₂, the thread B₁ is handling the most-urgent job (e.g.,oldest pending job).

At time t₅, the scheduling module 118 transfers the job J₃ from threadB₂ to the idle thread A₁ of CU₁. As a result, both jobs J₂ and J₃ nowrun on computing units that execute a single thread. The transfer of jobJ₃ to CU₁ therefore causes both jobs J₂ and J₃ to run faster. Thescheduling module 118 could have achieved the same effect bytransferring the most-urgent job J₂ to CU₁, instead of the job J₃. Buttransferring the job J₃ leaves the most-urgent job J₂ undisturbed, whichmay allow it to complete slightly faster (e.g., by not incurring aprocessing delay associated with the transfer).

FIG. 9 is a procedure 900 which generalizes the operations of FIG. 8 inflowchart form. In block 902, the scheduling module 118 determineswhether it has received a new job. If so, in block 904, the schedulingmodule 118 determines whether there is an idle computing unit (e.g., anidle processing core) available. If this is so, then in block 906, thescheduling module 118 assigns the new job to one of the idle computingunits using any selection policy (e.g., by randomly selecting an idlecomputing unit).

If block 904 is answered in the negative, then, in block 908, thescheduling module determines whether there is a thread available in acomputing unit, where that computing unit is currently executing atleast one other job in one of its threads. If this is true, then, inblock 910, the scheduling module 118 allocates the new job to ahighest-performing thread. A computing unit having thehighest-performing thread is a computing unit that is executing theleast number of threads.

Assume that there are two or more qualifying computing units that hosthighest performing threads. For example, in two-way SMT, assume that twocomputing units are each running one other thread. Or in four-way SMT,assume that two computing units are each running two other threads, etc.In this case, the scheduling module 118 can select one of the computingunits using the following illustrative and non-limiting logic: (a)first, the scheduling 118 identifies the most-urgent job currently beinghandled by each of the qualifying computing units, to provide a set ofmost-urgent jobs; (b) second, the scheduling module 118 identifies aleast-urgent job within the set of most-urgent jobs, providing a“least-urgent-among-most-urgent” job, e.g., a “youngest-oldest” job; (c)third, the scheduling module 118 identifies the computing unitassociated with the youngest-oldest job, to provide an identifiedcomputing unit; and (d) fourth, the scheduling module 118 assigns thenew job to an idle thread of the identified computing unit. Block 910summarizes this procedure by stating that the scheduling module 118selects the computing unit that is handling the “youngest-oldest” job.In the two-way SMT example of FIG. 8, the scheduling module 118 can bemore simply said to assign the new job to the computing unit handlingthe youngest job. In block 912, the scheduling module 118 stores the newjob in the job queue 120 if there are no idle threads available.

In block 914, the scheduling module 118 determines whether a job beingexecuted on a thread has ended for any reason. If so, in block 916, thescheduling module 118 determines whether there is an in-progress jobthat is being performed on a lower-performing thread (compared to theperformance level of the newly-vacated thread). If this is true, then,at block 918, the scheduling module 118 identifies the most-urgent jobthat is being performed by a lower-performing thread. That most-urgentjob will have one or more companion jobs running on the same computingunit. In one approach, the scheduling module 118 identifies theleast-urgent of those companion jobs and moves it to the newly-vacatedthread. However, in other implementations, the scheduling module 118 canapply other rules to determine which job to transfer to thenewly-vacated thread. Any job can be transferred such that the transferoperation results in the desired goal of improving the processing speedat which the most-urgent job is performed.

FIG. 9 indicates that the operations of blocks 916 and 918 can berepeated one or more times until no in-progress job can be moved to ahigher-performing thread. This job migration will result in a freed-upthread.

In block 920, after performing the job migration described above, thescheduling module 118 determines whether there are pending jobs in thejob queue 120. If so, in block 922, the scheduling module 118 moves themost-urgent job (e.g., oldest job) in the job queue 120 to the freed-upthread.

The scheduling module 118 can also incorporate the logic of FIG. 6 intoits processing operations. In this mode, the scheduling module 118 cantransfer a new job to a lowest-performing thread. If that job fails tocomplete within a prescribed amount of time, the scheduling module 118can transfer the job to a higher-performing thread. The schedulingmodule 118 can also incorporate the logic of FIG. 7.

The scheduling module 118 can perform the procedure 900 of FIG. 9subject to zero, one, or more job placement constraints. In a firstcase, the scheduling module 118 places no constraints on any computingunit that is capable of performing SMT. As such, that computing unit canrun any number of jobs in its threads. The processing system 102operating under this mode can be expected to have the followingbehavior. At suitably low traffic volume, the processing system 102 willprovide mostly high-performing processing modules (e.g., single threadsrunning on computing units). As traffic volume increases, the processingsystem 102 will increase the mix of processing modules having differentperformance levels. And at suitably high volumes, the processing system102 will provide all low-performing processing modules because,presumably, all the threads of the computing units will be constantlyoccupied in processing a heavy flow of job requests.

In a second case, the scheduling module 118 can define a quota thatlimits the number the jobs that can be processed by the computing unitsat any given time. For example, assume that the processing system 102includes six processing cores that each includes two hardware threads.This processing system can accommodate a total of 12 jobs at any onetime. But the scheduling module 118 may set the maximum job quota at,for example, 8 jobs. The advantage of this implementation is that thescheduling module 118 will thereby reserve a certain number of computingunits which offer higher-performing threads at any given time (which isnot true for the first-stated implementation). This, in turn, willensure that at least some of the resource-intensive jobs can be migratedup to higher-performing threads if need be. But the scheduling module118 otherwise places no limitations on which computing unit, at anygiven time, will offer the higher-performing threads.

In a third case, the scheduling module 118 can designate a maximumnumber of threads that each computing unit is permitted to use whenprocessing jobs. For example, in the six-core two-way SMT example statedabove, the scheduling module 118 can designate four cores as offeringtwo hardware threads, and the remaining two cores as offering onehardware thread (even though these cores can execute two threads). Thisachieves the same end result as stated above, because the cores canaccommodate a maximum of 8 jobs. But, in the third case, the schedulingmodule 118 can no longer dynamically choose which computing unitoperates using two threads. But even in this case, each computing unitthat runs two threads can be considered as offering dynamic performancebecause it can run using one or two threads, offering slower and fasterperformance levels, respectively.

More generally, all three implementations described above may offercertain advantages over a fixed collection of heterogeneous processingmodules. For example, in all cases, the SMT-enabled computing units arereconfigurable to some extent (e.g., by executing different numbers ofthreads). This enables the scheduling module 118 to dynamically adapt tochanges in the nature of job requests that are received over time, andchanges in the volume of such requests. This may have certainperformance advantages in certain circumstances over a fixedheterogeneous configuration, e.g., as measured in terms of averagequality, throughput, and/or energy efficiency; these advantages may bemore pronounced when the traffic that is received is highly variable.

Further, the scheduling module 118 can also dynamically alter itsconfiguration settings depending on the nature and volume of job trafficthat is received. It can do this by modifying the number of active jobsthat are permitted to be processed at any given time, and/or changingthe settings which determine how many threads each individual computingunit is allowed to execute. For example, assume that the schedulingmodule 118 observes that it is receiving a large quantity ofquickly-executable job requests. In this case, the scheduling module 118may wish to increase the job quota, e.g., from a maximum of 8 jobs to amaximum of 10 jobs in the above example. If the flow ofcomputation-intensive jobs increases, however, the scheduling module 118may wish to throttle back on the number of jobs that are permitted,e.g., from a maximum of 10 jobs to 6 jobs.

C. Representative Computing Functionality

FIG. 10 shows an illustrative environment 1000 in which the type ofprocessing system 102 shown in FIG. 1 can be employed. That is, FIG. 10shows a data center 1002 having resources that are provided at a singlelocation or distributed over plural locations. The resources include aplurality of processing systems (e.g., 1004, 1006, 1008, . . . ). Eachof these processing systems (1004, 1006, 1008) can be implemented usingthe functionality shown in FIG. 1. For example, each processing systemmay represent a server having plural processing cores. The processingcores may have different physical characteristics (e.g., differentmicroarchitectures), or may be driven under different conditions, etc.Or the processing cores may use SMT to execute one or more threads.Threads have different performance levels due to the factors describedabove, and can therefore be regarded as heterogeneous resources.

The data center 1002 can also include interconnection infrastructure1010 which connects the processing systems (1004, 1006, 1008) together.That interconnection infrastructure 1010 can include a hierarchicalnetwork of switches, for example.

Although not shown, the data center 1002 may also include an optionalcentral scheduling module. As explained in Section A, the centralscheduling module can replace or supplement the individual schedulingmodule 118 provided in each processing system.

A user may access the data center 1002 using any type of computerdevice, such as a personal computer, a smart phone, etc. FIG. 10represents these computer devices as job submission entities 1012. Theentities 1012 may interact with the data center 1002 via any type ofcommunication conduit 1014, such as a wide area network (e.g., theInternet), a local area network, a point-to-point connection, and so on.

In other cases, a user may interact with one or more local processingsystems which implement the principles set forth above. For example, alocal computer device may perform any application that utilizes a set ofheterogeneous processing modules provided by the local computer device.Still other environments can use the kind of heterogeneous processingsystem 102 described above.

FIG. 11 sets forth illustrative computing functionality 1100 that can beused to implement any aspect of the functions described above. Forexample, the type of computing functionality 1100 shown in FIG. 11 canbe used to implement any aspect of the processing system 102 of FIG. 1.For instance, the computing functionality 1100 may correspond to aserver or a local user device, etc. In all cases, the computingfunctionality 1100 represents one or more physical and tangibleprocessing mechanisms.

The computing functionality 1100 can include volatile and non-volatilememory, such as RAM 1102 and ROM 1104, as well as one or more processingdevices 1106 (e.g., one or more CPUs, and/or one or more GPUs, etc.).The processing device 1106 may implement two or more computing units,such as two or more processing cores. Each core, in turn, may implementone or more hardware threads and/or one or more software threads usingthe SMT technique. The computing functionality 1100 also optionallyincludes various media devices 1108, such as a hard disk module, anoptical disk module, and so forth. The computing functionality 1100 canperform various operations identified above when the processingdevice(s) 1106 execute instructions that are maintained by any memory(e.g., RAM 1102, ROM 1104, or elsewhere).

More generally, instructions and other information can be stored on anycomputer readable medium 1110, including, but not limited to, staticmemory storage devices, magnetic storage devices, optical storagedevices, and so on. The term computer readable medium also encompassesplural storage devices. In many cases, the computer readable medium 1110represents some form of physical and tangible entity. The term computerreadable medium also encompasses propagated signals, e.g., transmittedor received via physical conduit and/or air or other wireless medium,etc. However, the specific terms “computer readable storage medium” and“computer readable medium device” expressly exclude propagated signalsper se, while including all other forms of computer readable media.

The computing functionality 1100 also includes an input/output module1112 for receiving various inputs (via input devices 1114), and forproviding various outputs (via output devices 1116). Illustrative inputdevices include a keyboard device, a mouse input device, a touchscreeninput device, a gesture input device, a voice recognition mechanism, andso on. Illustrative output devices include a display device, a printer,a voice output device, and so on. The computing functionality 1100 canalso include one or more interfaces 1118 for exchanging data with otherentities inside a data center and/or external to the data center, e.g.,via one or more communication conduits 1120. One or more communicationbuses and/or component interconnection networks 1122 communicativelycouple the above-described components together.

The communication conduit(s) 1120 can be implemented in any manner,e.g., by a local area network, a wide area network (e.g., the Internet),etc., a direct component interconnection network, an indirect componentinterconnection network, and so on. The communication conduit(s) 1120can include any combination of hardwired links, wireless links, routers,gateway functionality, name servers, etc., governed by any protocol orcombination of protocols.

Alternatively, or in addition, any of the functions described in thepreceding sections can be performed, at least in part, by one or morehardware logic components. For example, without limitation, thecomputing functionality can be implemented using one or more of:Field-programmable Gate Arrays (FPGAs); Application-specific IntegratedCircuits (ASICs); Application-specific Standard Products (ASSPs);System-on-a-chip systems (SOCs); Complex Programmable Logic Devices(CPLDs), etc.

In closing, the description may have described various concepts in thecontext of illustrative challenges or problems. This manner ofexplanation does not constitute an admission that others haveappreciated and/or articulated the challenges or problems in the mannerspecified herein. Further, the claimed subject matter is not limited toimplementations that solve any or all of the noted challenges/problems.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

1-20. (canceled)
 21. A system comprising: a plurality of processingmodules having physical hardware, the plurality of processing modulesbeing characterized by performance levels reflecting hardwarecharacteristics of the physical hardware of the processing modules, theplurality of processing modules including at least two processingmodules having different hardware characteristics and differentperformance levels; and a scheduling module configured to: receive jobsassociated with varying respective service demands; and assign the jobsto individual processing modules based at least on the hardwarecharacteristics of the individual processing modules and the servicedemands associated with the jobs.
 22. The system of claim 21, whereinthe at least two processing modules comprise a first processing core ofa processing unit and a second processing core of the processing unit.23. The system of claim 22, wherein the first processing core and thesecond processing core of the processing unit comprise differenthardware components.
 24. The system of claim 22, wherein the firstprocessing core and the second processing core of the processing unitincorporate different processing architectures.
 25. The system of claim22, wherein the first processing core and the second processing core ofthe processing unit run at different speeds.
 26. The system of claim 22,wherein the first processing core and the second processing core of theprocessing unit run at different power budgets.
 27. A system comprising:one or more processing units, an individual processing unit includingmultiple processing cores with different physical characteristics,wherein the multiple processing cores include at least a firstprocessing core with first physical characteristics and a secondprocessing core with second physical characteristics; and at least onecomputer-readable storage medium storing scheduling instructions which,when executed by the one or more processing units, cause the one or moreprocessing units to: receive jobs associated having varying respectiveservice demands; and assign the jobs to the multiple processing cores ofthe individual processing unit based at least on the service demands ofthe jobs and the different physical characteristics of the multipleprocessing cores.
 28. The system of claim 27, wherein the servicedemands identify corresponding amounts of processing resources toperform the jobs in a specified period of time.
 29. The system of claim27, wherein the first processing core has a relatively higherperformance level than the second processing core.
 30. The system ofclaim 29, wherein the scheduling instructions, when executed by the oneor more processing units, cause the one or more processing units to:responsive to receipt of a new job: determine whether the firstprocessing core having the relatively higher performance level isavailable; and provided the first processing core having the relativelyhigher performance level is available, assign the new job to the firstprocessing core.
 31. The system of claim 29, wherein the schedulinginstructions, when executed by the one or more processing units, causethe one or more processing units to: responsive to receipt of a new job:determine whether the first processing core having the relatively higherperformance level is available; and when the first processing corehaving the relatively higher performance level is not available, assignthe new job to the second processing core.
 32. The system of claim 29,wherein the scheduling instructions, when executed by the one or moreprocessing units, cause the one or more processing units to: responsiveto a particular job ending on the first processing core, moving anin-progress job from the second processing core to the first processingcore.
 33. The system of claim 27, embodied as a single computing device.34. A method comprising: obtain jobs for processing; determiningprocessing module assignments of the jobs to multiple processing modulesbased at least on physical characteristics of the multiple processingmodules and ages of the jobs; and process the jobs according to theprocessing module assignments.
 35. The method of claim 34, wherein themultiple processing modules include a first core of a processing unitand a second core of the processing unit.
 36. The method of claim 35,wherein the first core of the processing unit runs at a first speed andthe second core of the processing unit runs at a second speed, whereinthe first speed is relatively faster than the second speed.
 37. Themethod of claim 36, further comprising: responsive to receipt of a newjob: determining whether the first processing core having the relativelyhigher first speed is available; and provided the first processing corehaving the relatively higher first speed is available, assigning the newjob to the first processing core.
 38. The method of claim 36, furthercomprising: responsive to receipt of a new job: determining whether thefirst processing core having the relatively higher first speed isavailable; and when the first processing core having the relativelyhigher first speed is not available, assigning the new job to the secondprocessing core.
 39. The method of claim 35, wherein the first core ofthe processing unit has a first processing architecture that provides afirst performance level and the second core of the processing unit has asecond processing architecture that provides a second performance level,wherein the first performance level is relatively higher than the secondperformance level.
 40. The method of claim 39, further comprising:responsive to receipt of a new job: determining whether the firstprocessing core having the first processing architecture that providesthe relatively higher first performance level is available; and providedthe first processing core having the first processing architecture thatprovides the relatively higher first performance level is available,assigning the new job to the first processing core.